Papers with open-source model
Are LLMs Good Annotators for Discourse-level Event Relation Extraction? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated proficiency in a wide array of natural language processing tasks, but their effectiveness over discourse-level event relation extraction tasks remains unexplored. |
| Approach: | They evaluate LLMs' ability to address discourse-level event relation extraction tasks using an open-source model and a commercial model. |
| Outcome: | The proposed model performs poorly on discourse-level event relation extraction tasks. |
Hindsight: Structured Agent Memory that Retains, Recalls, and Reflects (2026.acl-demo)
Copied to clipboard
Christopher Latimer, Nicolò Boschi, Andrew Neeser, Chris Bartholomew, Gaurav Srivastava, Xuan Wang, Naren Ramakrishnan
| Challenge: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Approach: | Hindsight organizes long-term memory into four logical networks and exposes three core operations. |
| Outcome: | Hindsight is a working memory system for AI agents that separates facts from beliefs . the system outperforms existing models on LongMemEval and LoCoMo with 83.6% accuracy . |
Arcee’s MergeKit: A Toolkit for Merging Large Language Models (2024.emnlp-industry)
Copied to clipboard
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, Jacob Solawetz
| Challenge: | Open-source language models can merge their parameters to improve performance and versatility without additional training. |
| Approach: | They propose to integrate model checkpoints into powerful multitask models without additional training. |
| Outcome: | the library has facilitated the merging of thousands of models, contributing to some of the world’s most powerful open-source model checkpoints. |
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant progress in utilizing tools, but their closed-source nature and high inference costs pose limitations on their adaptability. |
| Approach: | They propose a framework that performs task-solving by first creating a toolkit and then integrating the planning and calling of tools through a chain-of-solve approach. |
| Outcome: | The proposed model performs task-solving by harnessing Toolink's creativity and CoS ability on ChatGPT and finetunes the LLaMA-7B model. |
ProcessBench: Identifying Process Errors in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | Existing models fail to generalize to more challenging math problems, authors say . existing benchmarks related to assessing language models' reasoning process are limited . |
| Approach: | They propose a tool to measure language models' ability to identify erroneous steps in reasoning . they use two types of models: process reward models and critic models . |
| Outcome: | The proposed model outperforms existing models in evaluating language models' reasoning process . the best open-source model has demonstrated the critique capability competitive with the proprietary model . |
ViGPTQA - State-of-the-Art LLMs for Vietnamese Question Answering: System Overview, Core Models Training, and Evaluations (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) and their applications in low-resource languages are limited due to lack of training data and benchmarking datasets. |
| Approach: | They propose a question-response system for Vietnamese that uses LLMs . they propose to open-source the model and train it on benchmark datasets based on Vietnamese data . |
| Outcome: | The proposed question answering system for Vietnamese is open-source and performant . it can learn and capture human-like text, but there is a gap in evaluations for Vietnamese . |
Prompt Leakage effect and mitigation strategies for multi-turn LLM Applications (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Prompt leakage poses a compelling security and privacy threat in LLM applications. |
| Approach: | They propose a model which leverages the LLM sycophancy effect and a threat model which fine tunes an open-source model to defend against prompt leakage attempts. |
| Outcome: | The proposed model elevates the average attack success rate (ASR) from 17.7% to 86.2% in a multi-turn setting. |
CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinions (2025.findings-naacl)
Copied to clipboard
| Challenge: | CaseSumm is a dataset for long-context summarization in the legal domain . human groundtruth summaries are often not available for legal summarizing . |
| Approach: | They propose a dataset for long-context summarization that includes SCOTUS opinions and their official summaries. |
| Outcome: | The proposed dataset is the largest open legal case summarization dataset . it outperforms larger models on automatic metrics and human evaluation . |
LLMDet: A Third Party Large Language Models Generated Text Detection Tool (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing detection tools rely on access to LLMs and can only distinguish between machine-generated and human-authored text. |
| Approach: | They propose a model-specific, secure, efficient, and extendable detection tool that can source text from specific LLMs. |
| Outcome: | The proposed tool can source text from specific LLMs, such as GPT-2, OPT, LLaMA, and others. |
David vs. Goliath: Cost-Efficient Financial QA via Cascaded Multi-Agent Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated remarkable reasoning capabilities, but performance in FQA remains limited. |
| Approach: | They propose a low-cost yet effective framework that enables small LLMs to perform complex reasoning tasks without expensive models. |
| Outcome: | The proposed framework outperforms the best open-source model on BizBench by 10.46% and achieves competitive performance to GPT-3.5 using significantly fewer parameters. |
IHEval: Evaluating Language Models on Following the Instruction Hierarchy (2025.naacl-long)
Copied to clipboard
Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang
| Challenge: | Instruction-tuned language models (LMs) are increasingly deployed as interactive services across various applications. |
| Approach: | They propose a benchmark to evaluate models' ability to follow the instruction hierarchy by comparing their models to a set of benchmarks. |
| Outcome: | The proposed benchmark covers 3,538 examples across nine tasks covering cases where instructions in different priorities either align or conflict. |
LOLA – An Open-Source Massively Multilingual Large Language Model (2025.coling-main)
Copied to clipboard
Nikit Srivastava, Denis Kuchelev, Tatiana Moteu Ngoli, Kshitij Shetty, Michael Roeder, Hamada Zahera, Diego Moussallem, Axel-Cyrille Ngonga Ngomo
| Challenge: | Using a sparse Mixture-of-Experts Transformer architecture, our model is highly efficient and efficient across languages. |
| Approach: | They propose a multilingual large language model trained on more than 160 languages using a sparse Mixture-of-Experts Transformer architecture. |
| Outcome: | The proposed model performs well on natural language generation and understanding tasks while avoiding the common pitfalls of multilinguality. |
Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evidence that humans make numerous inferences to understand discourse and text is not fully understood. |
| Approach: | They propose to use textual inference datasets with multi-sentence premises to solve the entailment verification problem. |
| Outcome: | The proposed model outperforms GPT-3.5 and rivals GPL-4 in EV tasks. |
Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that Legal-R1 delivers competitive performance across diverse tasks. |
| Approach: | They propose to evaluate 12 large language models across 17 legal tasks across statutory and case-law traditions to determine their general reasoning performance. |
| Outcome: | The proposed model performs well across 17 legal tasks across statutory and case-law traditions. |
Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing instruction tuning datasets are limited by the quality of the instruction tuning data. |
| Approach: | They propose a model that converts unannotated text into task-specific training datasets for instruction tuning. |
| Outcome: | The proposed model improves the performance of pretrained and instruction tuned models over the de facto self-supervised baseline. |
CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models exhibit a specific cultural bias, neglecting values and differences of low-resource regions. |
| Approach: | They propose a culturally-aware training paradigm that leverages multilingual data and fine-grained reward modeling to enhance cultural sensitivity and inclusivity. |
| Outcome: | The proposed model achieves state-of-the-art in cultural alignment and general reasoning. |
Towards a Zero-Data, Controllable, Adaptive Dialog System (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent approaches to controllable dialog systems require additional training data to be deployed in new domains. |
| Approach: | They propose to generate dialog tree data directly from dialog trees by using a commercial Large Language Model or a single GPU. |
| Outcome: | The proposed approach can achieve comparable dialog success to models trained on human data. |
InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing LLMs break down on long-horizon tasks due to unbounded context growth and accumulated errors. |
| Approach: | They propose a framework that externalizes persistent state into a file-centric state abstraction and keeps the agent’s reasoning context strictly bounded regardless of task duration. |
| Outcome: | Experiments on DeepResearch and an 80-paper literature review show that the proposed framework maintains higher long-horizon coverage than baseline models without task-specific fine-tuning. |